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EXECUTIVE  SUMMARY 


The  main  objective  of  the  overall  proposed  research  is  to  develop  a  robust  video  communication 
over  Airborne  Networks  that  are  operating  under  time  varying  and  error  prone  battlefield 
environments  with  limited  bandwidth  and  potentially  transmitting/receiving  with  mobile  and  portable 
devices.  The  overall  robust  video  communication  system  aiming  at  providing  high  performance  video 
transmission  under  dynamic  and  time  varying  Airborne  Networks  will  be  explained  in  the  technology 
background  section.  The  objective  of  the  first  phase  of  this  project  is  to  develop  a  channel  simulation 
toolset  that  is  able  to  capture  some  major  characteristics  of  the  Airborne  Networks. 

During  the  first  12  months  of  this  project  period,  the  team  at  the  Florida  Institute  of  Technology 
has  been  able  to  implement  a  wireless  channel  simulation  based  on  Finite-State  Markov  Model  to 
simulate  and  analyze  the  error  and  loss  characteristics.  This  is  a  crucial  step  towards  the  overall  robust 
video  transmission  system  to  be  developed  specifically  for  Airborne  Networks.  By  adopting  the 
Finite-State  Markov  Model,  we  have  also  been  able  to  obtain  relatively  accurate  estimation  of  the 
channel  parameters  based  on  the  training  of  the  simulated  channel  loss  with  simplified  Markov  Model. 
The  simulation  is  currently  based  on  command  line  execution. 

As  this  is  only  the  first  phase  of  the  proposed  overall  system,  the  team  is  working  on  several  next 
steps  to  refine  the  channel  modeling  with  multi-stage  Markov  models  and  with  more  accurate 
estimation  of  channel  parameters.  The  team  is  also  working  on  the  GUI  (graphic  user  interface)  for  the 
simulation  toolset  for  convenient  user  interface.  This  final  report  for  the  first  year  of  the  project 
summarizes  the  activities  completed  from  January  2007  to  December  2007. 

1.  Technology  Background 

1.1.  Contemporary  Enabling  Technologies 

Under  today’s  Global  Information  Grid  initiative,  the  Air  Force  is  developing  a  contemporary  Airborne 
Networking  technology  to  meet  the  challenges  of  secure  and  reliable  communication  of  battlefield  critical 
information  within  the  air-ground-space  domains  of  military  personnel  and  devices.  Wireless  networking, 
by  its  very  own  nature,  is  already  facing  significant  problem  due  to  time  varying  characteristics  of  the  links 
and  the  constant  moving  of  the  receiving  devices.  However,  Airborne  Networking  will  face  several 
additional  challenges  because  of: 

(1)  Platform  mobility  is  in  the  “mach”  not  “mph”  in  the  ground  networking  scenarios 

(2)  Latency  requirement  is  measured  in  “millisecond”  not  “second”  in  other  applications 

(3)  Aircraft  antennas  usually  have  much  less  gains 

(4)  Increasing  demand  for  video  over  airborne  networks. 

The  requirement  for  significantly  more  video  communication  over  Airborne  Networks  poses  substantial 
challenges  for  the  development  of  next  generation  Airborne  Networks  as  many  of  the  legacy  airborne 
platforms  are  inherently  running  at  low  bit  rate.  The  next  generation  Airborne  Networks  will  need  to  ensure 
the  smooth  transitions  from  current  legacy  platforms  to  future  robust  networks. 

Recent  technological  advances  in  the  wireless  communication  and  networking  at  the  commercial  front 
have  enabled  numerous  applications  based  on  mobile  wireless  networking.  Currently,  there  are  several 
commercially  operational  wireless  networks  enables  seamless  roaming  of  consumers  across  these  wireless 
networks  and  offers  unprecedented  complexity  of  integrated  services,  ranging  from  voice,  data,  to  graphics 
and  video.  In  particular,  significant  research  and  development  activities  have  been  invested  to  develop 
applications  for  the  streaming  of  multimedia  data  over  wireless  networks  to  a  wide  range  of  mobile 
portable  devices. 

On  the  other  hand,  the  infrastructure  development  in  wireless  networks  has  been  duly  complemented 
by  the  tremendous  advances  in  multimedia  encoding  technologies.  In  fact,  the  success  of  wireless 
multimedia,  especially  wireless  video,  in  the  consumer  electronics  sector,  has  been  made  possible  by  the 
successful  development  of  a  series  of  video  coding  standards:  MPEG-1,  MPEG-2,  MPEG-4,  H.263,  and  the 
most  recent  one  H.264.  The  evolution  of  these  video  coding  standards  has  spanned  over  several  decades 
since  the  1980’s  and  has  gone  through  several  generations  of  development.  Over  the  past  two  decades, 
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these  video  coding  standards  allow  high  performance  compression  of  video  data  so  as  to  match  with  the 
limited  bandwidth  of  wireless  links  for  video  transmission. 

In  addition  to  meeting  the  bandwidth  compression  requirements,  recent  video  coding  standards,  such 
as  MPEG-4  and  H.264,  also  include  a  suite  of  error  resilient  tools  in  order  to  minimize  the  error  impact 
from  unreliable  wireless  links.  Furthermore,  to  facilitate  the  heterogeneous  wireless  systems  and  mobile 
devices,  both  MPEG-4  and  H.264  provide  scalable  coding  profile  so  that  the  compressed  video  bitstream 
can  be  truncated  at  any  bit  rate  to  fit  the  heterogeneous  bandwidth  requirements  of  different  wireless  links 
and  portable  devices.  It  is  the  error  resilient  and  scalable  video  coding  strategy  embedded  in  H.264  standard 
that  matches  well  with  the  mission  of  video  transmission  task  within  the  Air  Force  Airborne  Networks. 

In  summary,  the  contemporary  video  coding  standards  are  able  to  offer  the  following  major  benefits  for 
robust  video  transmission  over  error  prone  Airborne  Networks: 

(1)  Significant  compression  capability  to  meet  the  bandwidth  constraints 

(2)  Error  resilient  capability  to  meet  the  error  robust  transmission  requirement 

(3)  Scalable  video  profile  to  seamlessly  meet  the  variable  bandwidth  of  heterogeneous  wireless 
links 

1.2.  From  Commercial  Technology  to  Airborne  Networking  Applications 

It  is  true  that  the  commercial  wireless  networks  have  achieved  great  success  during  the  last  decade, 
with  deep  penetration  of  cellular  phones  and  mobile  devices  to  hundreds  millions  of  population  worldwide. 
Much  of  the  success  is  due  to  our  fill  understanding  of  the  wireless  transmission  media  that  enables  the 
design  of  high  performance  networks  and  handheld  devices  such  as  cellular  phones. 

There  is  no  doubt  that  the  success  of  these  commercial  applications  offers  great  lessons  for  developing 
video  transmission  system  over  Airborne  Networks.  However,  major  challenges  remain  in  our  attempt  to 
extend  the  commercial  applications  to  military  battlefield  scenarios.  One  of  these  significant  challenges  is 
that  the  wireless  networks  in  Airborne  Platforms  are  fundamentally  different  from  the  commercial  ones,  in 
their  speed  of  mobility,  latency  requirement,  antenna  gains,  and  hostile  environments.  The  lessons  we 
learned  in  the  development  of  commercial  systems  cannot  be  directly  applied  to  the  airborne  networking 
applications  because  of  these  fmdamental  differences. 

To  develop  a  robust  video  communication  system  deployable  in  the  Airborne  Platforms,  we  need  to 
first  carefully  investigate  the  characteristics  of  the  wireless  links  within  the  Airborne  Networking  scenarios. 
Once  we  have  the  comprehensive  knowledge  of  these  unique  wireless  links,  we  can  follow  the  successful 
strategies  in  the  development  of  commercial  wireless  video  systems  to  design,  develop,  and  deploy  the 
robust  video  communication  systems  in  the  Airborne  Networking  platforms. 

It  is  clear  that  the  first  step  to  address  the  differences  between  commercial  wireless  links  and  the 
Airborne  networking  links  will  to  develop  mathematically  tractable  models  for  various  Airborne  Network 
links.  It  is  with  this  consideration  that  we  dedicate  the  first  phase  of  the  proposed  robust  video 
communication  system  to  the  modeling  of  such  wireless  links  via  the  Finite-State  Markov  Model.  We 
believe  that  such  modeling  will  be  able  to  offer  some  useful  insight  to  design  of  the  over  video 
communication  system  for  Airborne  Networks. 

In  this  report,  we  will  first  present  the  brief  overview  of  the  proposed  robust  video  communication 
system.  We  will  then  focus  on  the  report  on  the  first  phase  of  the  proposed  overall  system,  that  is,  the 
implementation  of  the  Finite-State  Markov  Modeling  of  the  wireless  links  and  the  proposed  estimation  of 
the  wireless  channels  based  on  such  modeling. 

2.  Robust  Video  Communications  System  Overview 

The  proposed  robust  video  communications  system  will  be  designed,  developed,  and  implemented  by 
the  collaboration  between  the  research  team  at  the  Florida  Institute  of  Technology  and  the  researchers  at  Air 
Force  Rome  Research  Lab.  This  system  aims  at  providing  high  quality  video  transmission  over  severely 
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impaired  wireless  links  between  nodes  that  are  connected  within  the  airborne  networks.  The  target  bit  rate 
for  the  proposed  video  communication  can  be  in  the  range  between  24  Kbps  and  384  Kbps  with  relatively 
high  visual  quality.  However,  we  may  allow  the  system  to  operate  at  extreme  low  bit  rate  down  to  10  Kbps 
and  at  high  bit  rate  up  to  1.5  Mbps  so  that  we  are  still  able  to  push  the  video  communication  for  some 
legacy  communication  networks  and  devices.  To  accommodate  the  large  range  of  the  data  rate  for 
heterogeneous  wireless  links  and  devices,  we  adopt  the  H.264  SVC  standard  for  video  coding  and  decoding. 
To  overcome  the  adversity  of  the  video  transmission  in  tactical  airborne  wireless  networks  with  various 
constraints  in  terms  of  bandwidth,  channel  impairments,  both  bursty  errors  and  packet  loss,  and  media 
access  control  for  battery  operated  end  users,  we  propose  an  end-to-end  design  principle  that  embrace  the 
joint  source  and  channel  coding  as  the  key  strategy  and  the  channel  estimation  and  feedback  as  the  means 
of  providing  adaptation.  Various  building  blocks  of  the  proposed  system  and  the  relationship  between  these 
building  blocks  are  shown  in  Figure  1. 


Figure  1:  Building  blocks  for  the  proposed  robust  video  communication  system 

In  the  following,  we  will  present  an  overview  of  the  proposed  robust  video  communication  system  and 
will  briefly  introduce  several  major  building  blocks  and  their  functionalities  and  relationships.  Detailed 
description  of  all  building  blocks  and  design  principles  will  be  presented  in  the  next  Section. 


Major  Functionalities  of  the  Proposed  System 

The  proposed  system  can  be  divided  into  three  major  functionalities:  (1)  video  source  encoding  and 
decoding,  (2)  error  correction  channel  encoding  and  decoding,  and  (3)  channel  modeling  and  estimation. 
Since  it  has  been  well  known  that  the  joint  source  and  channel  coding  is  able  to  significantly  improve  the 
end-to-end  quality  of  service  for  video  transmission,  these  functional  components  can  no  longer  be  treated 
separately.  This  is  evident  form  Figure  1  that  many  of  these  building  blocks  are  interconnected  to  facilitate 
joint  design  of  the  end-to-end  system.  We  describe  in  the  following  these  three  major  functionalities  and  the 
strategies  we  take  to  implement  these  functionalities. 

•  Video  Encoding  and  Decoding:  We  will  employ  H.264  SVC  standard  video  coding  as  a  base 
for  video  encoding  and  decoding.  Such  a  selection  of  standard  video  codec  will  enable  the 
smooth  exchange  of  motion  imagery  among  different  DoD  agencies  as  envisioned  by  the 
MISB.  In  addition  to  this  important  consideration,  H.264  SVC  video  encoding  and  decoding 
schemes  are  inherently  able  to  facilitate  the  required  error  control  strategies,  including  error 
resilience  tools,  data  partition  for  unequal  error  protection,  and  error  concealment. 
Significantly  improved  results  have  been  obtained  by  joint  source  and  channel  coding  based 
on  MPEG-4  codec  for  video  delivery  over  both  packet  loss  network  and  wireless  fading 
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channels.  We  will  extend  the  previously  developed  MPEG-4  codec  to  H.264  SVC  codec  for 
this  project. 

•  Channel  Encoding  and  Decoding:  The  main  reason  for  the  incorporation  of  channel 
encoding  and  decoding  is  that  the  communication  links  between  platform  nodes  within 
Airborne  Networks  can  be  severely  impaired.  The  embedded  error  resilience  in  H.264  SVC 
video  codec  is  inadequate  to  overcome  such  channel  impairment.  Additional  error  control 
strategies  need  to  be  implemented.  We  will  carefully  select  a  combination  of  error  control 
codes  for  the  proposed  system  because  in  order  to  combat  the  bursty  error  in  the  wireless  links 
as  well  as  the  packet  loss  error  due  to  networking  management.  In  combination  with 
intelligent  interleaving,  and  the  error  resilient  MPEG-4  tools,  a  joint  design  of  channel  coding 
and  source  coding  has  been  shown  to  achieve  high  end-to-end  quality  of  service  for  composite 
channel  impairments  with  both  packet  loss  and  burst  fading  errors.  We  will  extend  this 
innovative  approach  to  H.264  based  joint  source  and  channel  coding  schemes. 

•  Channel  Modeling,  Estimation,  and  Feedback:  The  design  issues  related  to  these 
functionalities  will  be  crucial  to  the  success  of  the  end-to-end  system  design.  The  joint  source 
and  channel  coding  will  rely  on  these  functionalities  to  provide  channel  feedback  information 
for  an  optimal  allocation  of  bit  budget  for  source  coding  and  channel  coding.  Since  the 
network  links  are  changing  constantly  while  there  is  latency  between  the  channel  feedback 
and  algorithm  adaptation,  there  is  a  need  for  the  trade-off  between  accuracy  and  speed  of  the 
channel  estimation.  We  will  first  investigate  the  fast  estimation  of  partial  channel  information 
and  develop  the  corresponding  adaptation  schemes,  and  then  strive  to  estimation  and  feedback 
the  complete  channel  information  for  joint  source  and  channel  coding  adaptation.  An 
additional  functionality  of  the  channel  feedback  is  to  decide  whether  or  not  the  transcoding  or 
scalable  coding  needs  to  be  activated  in  order  to  reach  certain  edge  users  or  edge  devices  that 
have  considerably  different  bandwidth  than  the  majority  of  the  platforms  within  Airborne 
Networks. 

In  summary,  these  major  functionalities  cover  all  the  building  blocks  within  the  proposed  video 
communications  system.  Several  building  blocks  serve  across  functionality  boundaries  and  operate  as 
platforms  for  joint  design.  One  example  is  the  JSCC  block  that  serves  across  source  coding  and  channel 
coding  boundaries.  The  JSCC  also  interact  intimately  with  the  channel  modeling,  estimation,  and  feedback 
blocks  that  bridge  between  the  transmitting  ends  with  receiving  ends. 

3.  Technical  Rationales  for  Finite-State  Markov  Modeling 

The  ultimate  goal  in  robust  video  communication  system  design  is  to  control  and  optimize  the  end-to- 
end  performance  adaptively  according  to  the  instantaneous  channel  conditions  of  the  communication  links. 
In  the  civil  and  commercial  wireless  communication  applications,  extensive  studies  on  the  channel  and 
networking  characteristics  have  been  carried  out  and  the  design  of  communication  systems  can  be 
optimized  based  on  the  instantaneous  feedback  of  the  channel  information  or  the  statistical  behavior  of  the 
channel  characteristics.  To  develop  a  robust  video  communication  system  for  airborne  networks,  the  first 
task  will  be  to  model  and  simulate  the  channel  characteristics  of  these  dynamic,  ad  hoc,  and  often  hostile 
wireless  links.  Once  we  have  collected  adequate  information  on  the  channel  characteristics  of  these  links, 
we  shall  be  able  to  apply  the  joint  source  and  channel  coding  principles  that  have  been  successfully 
employed  in  commercial  applications  to  the  Air  Force  Airborne  Networks. 

There  are  several  key  technologies  that  are  required  in  order  to  successfully  develop  the  proposed 
robust  video  communication  systems.  We  will  first  focus  on  the  channel  modeling  and  simulation  in  order 
to  provide  the  basic  information  for  the  subsequent  adaptive  joint  source  and  channel  coding  schemes.  We 
will  then  outline  the  strategy  for  channel  bandwidth  or  data  rate  regulation  to  achieve  the  desired  adaptation 
to  optimize  the  video  transmission  performance.  Such  adaptation  is  based  on  the  well-known  H.264  SVC 
standard  as  well  as  the  possible  punctured  channel  coding  techniques. 

In  this  report,  we  will  focus  on  the  channel  modeling  and  simulation  based  on  Finite-State  Markov 
Modeling.  Once  the  tools  for  such  modeling  are  developed,  a  systematic  investigation  of  the  modeling  and 
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simulation  can  be  carried  out  and  the  results  will  be  useful  for  the  design  and  development  to  proposed 
robust  video  communication  systems. 

1.3.  Wireless  Channel  Modeling  Using  Finite-State  Markov  Model 

In  the  project,  the  first  order  Finite-state  Markov  Model  (FSMM)  has  been  employed  to  characterize 
the  error  and  loss  behavior  of  dynamic  wireless  network  and  its  implementation.  In  the  following,  the 
FSMM  principles  will  be  outlined  and  the  procedure  to  train  and  estimate  the  wireless  links  will  also 
be  presented 


3.3.1.  Finite-sate  Markov  Model  (FSMM) 


A  finite-state  Markov  model  is  defined  as  a  finite  set  of  states  S  =  {.S’(l ,  .S’,  ,•  •  •  AA-i  }  >  where  K  is  the 
number  of  states.  A  binary  symmetric  channel  (BSC)  with  a  given  crossover  probability  is  associated 
with  each  state  Sk, k  e  {0,1,2,- •  ■} ,  so  the  channel  quality  for  each  state  can  be  identified.  Let 

{SH\SaeS,n  =  0,1,2  9  •  •  •}  to  be  a  constant  Markov  process.  Since  the  constant  Markov  process 
has  the  property  of  stationary  transitions,  the  transition  probability  is  independent  of  time  index  n ,  so 
tj,k  =  Pr{S„+,  =  sk  |  s„  =  s,}  k,j  €  0,1,2,  A  - 1  (1) 

is  a  constant  KxK  matrix,  which  is  called  the  state  transition  probability  matrix  T  .  The  state 
transition  probability  matrix  is  an  important  parameter  for  a  finite-state  Markov  model.  Note  that  the 
sum  of  the  elements  on  each  row  of  T  is  equal  to  1 . 

=l,V*e  {0,1,2,-}  (2) 

/=() 

Another  parameter  for  a  finite-state  Markov  model  is  the  crossover  probabilities  in  each  state, 

ek,k<={  0,1,2,  •••}. 

The  last  parameter  for  a  finite-state  Markov  model  is  the  steady  state  probability  of  each  state, 
pk,k<={  0,1,2,-}  But  the  steady  state  probabilities  are  not  independent.  The  steady  state 


probability  pk  for  state  Sk  can  be  viewed  as  the  probability  of  from  all  states  transition  to  state  Sk  ,  so 


K- 1 


./•=» 


=  pk,Vkz{  0,1,2,  ,*-1} 


K- 1 

The  functions  only  have  zero  solution.  But  ^  pk  =  1 

k= o 


(3) 


(4) 


With  any  K  —  1  functions  from  (3)  and  the  function  (4),  the  pk ,  k  e  {0,1,2,- ••}  can  be  determined 
with  any  given  state  transition  probability  matrix  T  . 


3.3.2.  Training  A  Finite-state  Markov  Model 

To  use  the  finite-state  Markov  model  to  model  a  dynamic  wireless  communication  channel,  one 
should  decide  the  parameters  of  a  finite-state  Markov  model.  We  call  the  process  of  using  a  series  of 
known  data  to  estimate  the  parameters  of  a  finite-state  Markov  mode  training  a  finite-state  Markov 
model. 


After  determine  the  parameters  of  a  finite-state  Markov  mode,  one  generate  a  series  of  states,  which 
hopefully  can  model  the  characteristics  of  the  modeled  wireless  channel. 
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1.4.  Implementation  of  Finite-State  Markov  Model 

We  use  C++  implement  the  any  number  of  states  finite-state  Markov  model,  which  is  defined  as  class 
as  following: 
class  FSMC 
{ 

public: 

FSMC(int  num  =  2);  //construction  function,  with  num  as  the  number  of  states,  default  is  2. 
~FSMC(); 

void  set_tran_matrix(double  * matrix );  //set  the  transition  matrix 
void  set_state _prob(double  *s _p);  //set  the  steady  state  probabilities 

void  set _pe(double  *p_e);  //set  the  crossover  probabilities 

char  oneStep();  //generate  the  next  state 

void  fsm  _train(char  *data,int  train_len);  //use  data  to  train  a  finite-state  Markov  model 
void  show();  //show  the  Markov  model 

private: 

int  num_state; 
int  current jstate; 
double  **tran_matrix, 
double  *state _prob; 
double  *pe; 

}; 

Since  we  don’t  have  data  right  now,  so  we  use  a  known  finite-state  Markov  model  to  generate  a  series 
artificial  data  and  use  those  data  to  train  a  new  finite-state  Markov  model.  The  trained  model  should 
have  the  same  parameters  with  the  given  finite-state  Markov  model. 

The  software  right  now  is  the  command  line  form,  which  takes  an  input  of  a  finite-state  Markov  model, 
to  train  a  now  finite-state  Markov  model,  show  and  compare  the  given  and  trained  models. 

How  to  use: 

Since  the  transition  matrix  can  detennine  the  steady  state  probabilities,  so  we  only  need  input,  so  we 
only  need  to  input  the  transition  matrix  for  the  given  model.  The  software  takes  input  in  two  ways: 

1.  The  program  read  the  transfer  matrix  from  a  file  (for  example  "data.txt"): 

input  "fsm  data.txt"  in  command  line 

2.  input  the  elements  of  transfer  matrix  from  command  line  one  by  one: 

input  "fsm"  in  command  line 

For  example,  we  have  following  data  in  file  “data.txt”  (we  assume  the  state  can  only  transfer  to  itself 
or  adjacent  states) 

4 

0.04  0.96  0  0 

0.04  0.92  0.04  0 
0  0.06  0.92  0.02 

0  0  0.05  0.95 

The  number  in  the  first  line  is  the  number  of  states,  followed  by  the  state  transition  matrix.  If  we  input 
"fsm  data.txt "  in  command  line,  we  can  get  the  following  results: 

The  original  Finite  State  Markov  Chain: 

Transfer  Matrix: 

0.04  0.96  0  0 


//number  of  states 
//current  state 

//transition  matrix 
//  steady  state  probabilities 
//  crossover  probabilities 
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0.04  0.92  0.04  0 

0  0.06  0.92  0.02 

0  0  0.05  0.95 

Steady  State  probability: 

0.0210971  0.506329  0.337553  0.135021 


The  Trained  Finite  State  Markov  Chain: 

Transfer  Matrix: 

0.0423045  0.957696  0  0 

0.0407675  0.919227  0.0400053  0 

0  0.0597448  0.920473  0.0197793 

0  0  0.0504601  0.94954 

Steady  State  probability: 

0.021558  0.506433  0.339092  0.132917 

We  can  see  from  the  results  that  the  trained  model  almost  have  the  same  parameters  with  the  given 
model. 


1.5.  Channel  Estimation  Using  FSMM 

We  propose  the  following  channel  state  estimation  at  the  decoder.  The  decoder  estimates  the 
channel  state  for  the  next  one  or  several  transmissions  (according  to  application  latency)  based  on  the 
current  received  packet  and  the  packet  after  channel  decoding.  Each  video  packet  is  protected  by 
different  strong  level  of  FEC,  depending  on  the  important  of  video  coding  layer  and  the  current 
channel  state.  After  receiving  one  packet,  the  decoder  first  does  channel  decoding. 

Based  on  the  received  original  packet  and  the  packet  after  channel  decoding,  the  channel  estimation 
method  is  outlined  as  following: 

•  If  the  channel  decoding  is  successful,  we  can  compare  the  received  original  packet  and  the  packet 
we  get  after  decoding.  The  different  number  of  bits  should  be  the  number  of  transmission  error. 
The  number  of  transmission  error  divided  by  the  length  of  the  packet,  is  the  error  probability  pe . 

•  If  current  state  is  good,  then  the  next  state  is  estimated  as  good; 

Else  if  current  state  is  bad: 

If  pe  <  e0  (the  crossover  probabilities  of  good  state),  then  the  next  state  is  estimated  as  good; 
Else,  the  next  state  is  estimated  as  bad. 

•  If  channel  decoding  is  unsuccessful,  then  the  next  state  is  estimated  as  bad. 


Currently,  the  channel  state  estimation  only  supports2  states  case.  The  simulation  is  based  on  Reed- 
Solomon  code  as  following: 
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n=255,  t  =  NUM  ERR  GOOD/  NUM  ERR  BAD 

NUM  ERR  GOOD  and  NUM  ERR  BAD  are  two  macros  defined  in  fsmApp.cpp,  which  stands  for 
number  of  bit  error  per  frame  in  good  channel  state  and  bad  channel  state  respectively. 

1.6.  Summary  and  Discussion 

In  summary,  this  report  outlines  the  modeling  and  simulation  of  wireless  channels  in  order  to 
characterize  the  dynamics  of  the  Airborne  Networks  links.  Initial  investigations  show  that  the 
proposed  modeling  based  on  finite-state  Markov  model  is  able  to  capture  some  key  features  of  the 
channel  with  transient  probability  for  state  transitions.  If  the  funding  for  the  second  year  and  beyond  is 
continued,  the  team  will  continue  the  investigation  and  hope  to  capture  more  key  features  of  the 
wireless  links  in  Airborne  Networking. 

The  next  steps  of  investigation  may  include  the  following: 

U  Expand  the  initial  model  to  include  more  states  within  the  FSMM  so  that  more  transient 
characteristics  can  be  capture 

□  Investigate  more  robust  estimation  algorithms  so  the  estimation  of  channel  conditions  can  be 
more  accurate  and  faster 

□  Work  on  the  implementation  of  graphics  user  interface  by  working  with  Air  Force  Rome  Lab 
technical  contact  so  that  the  simulation  can  be  more  user-friendly. 

4.  Software  Packages 


The  software  package  generated  out  of  this  research  project  will  be  included  in  a  separate  CD 
which  will  be  sent  to  AFRL  for  deposit.  In  addition  to  the  software  package  for  Finite-State  Markov 
Modeling,  this  team  has  also  modified  the  H.264  Scalable  Video  Coding  (SVC)  software  used  to 
integrate  the  proposed  robust  video  communication  system.  This  SVC  software  package  is  also 
included  in  the  submission. 
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on  Multimedia  Systems  and  Applications  X,  Proc.  of  SPIE  edited  by  Susanto  Rahardja,  Jong  Won 
Kim,  Jiebo  Luo,  Vol.  6777,  pages  677702-1  -  677702-8. 
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ABSTRACT 

In  this  paper,  a  wireless  channel  is  viewed  as  a  heterogeneous  network  in  the  time  domain,  and  an  adaptive  video 
transmission  scheme  for  H.264  scalable  video  over  wireless  channels  modeled  as  a  finite-state  Markov  chain  processes  is 
presented.  In  order  to  investigate  the  robustness  of  adaptive  video  transmission  for  H.264  scalable  video  over  wireless 
channels,  statistical  channel  models  can  be  employed  to  characterize  the  error  and  loss  behavior  of  the  video 
transmission.  Among  various  statistical  channel  models,  a  finite-state  Markov  model  has  been  considered  as  suitable  for 
both  wireless  links  as  Rayleigh  fading  channels  and  wireless  local  area  networks  as  a  combination  of  bit  errors  and 
packet  losses.  The  H.264  scalable  video  coding  enables  the  rate  adaptive  source  coding  and  the  feedback  of  channel 
parameters  facilitates  the  adaptive  channel  coding  based  on  the  dynamics  of  the  channel  behavior.  As  a  result,  we  are 
able  to  develop  a  true  adaptive  joint  source  and  channel  based  on  instantaneous  channel  estimation  feedback.  Preliminary 
experimental  results  demonstrate  that  the  estimation  of  the  finite-state  Markov  channel  can  be  quite  accurate  and  the 
adaptive  video  transmission  based  on  channel  estimation  is  able  to  perform  significantly  better  than  the  simple  channel 
model  in  which  only  average  bit  error  rate  is  used  for  joint  source  and  channel  coding  design. 


Keywords:  H.264,  Scalable  video  coding,  finite-state  Markov  chain,  channel  state  estimation,  unequal  error  protection 


1.  INTRODUCTION 

The  demand  for  high-quality  mobile  wireless  communication  services  (multimedia  broadcasting,  video  streaming, 
video  telephony,  etc)  beyond  conventional  voice  communication  is  increasing  at  an  explosive  rate.  However,  the  time- 
varying  characteristics  of  wireless  channel  still  remain  a  significant  obstacle  for  reliable  wireless  communication.  From 
the  link  layer  point  of  view,  the  time-vaiying  characteristic  of  the  wireless  channel  is  its  dynamic  bit  error  and  packet 
over  time.  From  application  layer  point  of  view,  its  bandwidth  is  dynamic  with  time,  so  wireless  channel  can  be  viewed 
as  a  heterogeneous  network  in  time  domain.  Scalable  video,  designed  for  heterogeneous  networks,  can  be  used  with  a 
wireless  channel  to  adapt  the  time  variance  in  bit  error  and  packet  loss. 

The  scalable  extension  of  H.264/MPEG4-AVC  is  a  current  standardization  project  of  the  Joint  Video  Team  (JVT)  of 
the  ITU-T  Video  Coding  Experts  Group  (VCEG)  and  the  ISO/IEC  Moving  Picture  Experts  Group  (MPEG).  This 
amendment  to  the  H.264  standard  is  expected  to  be  finalized  at  the  24th  JVT  meeting  29  June  2007.  Most  components  of 
H.264/MPEG4-AVC  are  used  as  specified  in  the  standard.  The  base  layer  of  an  SVC  bit-stream  is  generally  coded  in 
compliance  with  H.264/MPEG4-AVC,  and  each  standard  conforming  H.264/MPEG-4  AVC  decoder  is  capable  of 
decoding  this  base  layer  representation  when  it  is  provided  with  an  SVC  bit-stream.  The  experimental  results  show  that 
coding  efficiency  of  H.264  scalable  video  coding  is  much  better  than  previous  scalable  video  coding  (such  as  MPEG-4 
FGS).  For  SNR  scalability,  the  performance  of  H.264  scalable  video  coding  is  very  close  to  that  of  single  layer  coding 
[11- 
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In  this  paper,  we  investigate  the  new  H.264  scalable  video  coding  on  an  error  prone  wireless  channel.  As  we  all  know, 
due  to  multi-path,  mobility,  terrain  and  a  lot  of  other  complicated  reasons,  the  wireless  channel  are  unreliable  and 
dynamic.  In  order  to  investigate  the  performance  of  adaptive  video  transmission  for  H.264  scalable  video  over  the 
wireless  channels,  we  model  the  channel  as  a  finite-state  Markov  model.  Finite-state  Markov  modeling  of  a 
communication  channel  is  a  simple  and  effective  approach  for  communication  channel  description  [2].  To  transmit 
scalable  video  over  heterogeneous  networks,  the  encoder  must  know  the  capacity  of  each  link  in  order  to  deliver  a 
suitable  video  stream  (suitable  base  layer  and  enhancement  layer  descriptions)  to  each  link.  This  video  stream  can  make 
sure  the  decoder  at  each  link  can  get  the  best  quality  of  video  under  the  given  link  capacity.  When  applied  to  the  wireless 
channel,  i.e.,  the  time  domain  heterogeneous  network,  the  encoder  should  know  the  channel  state  at  the  each 
transmission  time.  If  the  channel  is  good,  the  encoder  can  generate  a  better  description  and  use  lighter  FEC  protection;  if 
the  channel  is  not  so  good,  the  encoder  can  generate  a  suitable  description  and  use  stronger  FEC  protection.  So  channel 
estimation  is  a  very  important  task  in  delivering  H.264  scalable  video  coding  over  the  wireless  channel.  We  propose  a 
channel  estimation  method  based  on  the  decoding  process  in  the  decoder,  and  feed  back  the  channel  state  to  the  encoder 
to  help  the  encoder  to  deliver  best  quality  video  based  on  the  estimated  channel  condition. 

The  rest  of  this  paper  is  organized  as  follows:  In  Section  2,  an  overall  system  description  of  the  proposed  scheme  is 
given.  We  will  also  discuss  both  scalable  H.264/AVC  codec  and  channel  state  estimation  based  on  finite-state  Markov 
Chain.  In  Section  3,  experimental  results  are  reported  to  demonstrate  that  the  proposed  scheme  indeed  is  able  to  perform 
significantly  better  than  the  non-rate  adaptive  system.  Section  4  concludes  this  paper  with  summary  and  discussion. 


2.  SYSTEM  DESCRIPTION 


The  proposed  scalable  video  transmission  over  a  time -varying  wireless  system  is  shown  in  Figure  1.  At  the  encoder 
side,  base  and  enhancement  layers  are  generated  by  scalable  extension  of  H.264/AVC  [3].  With  the  feedback  of 
estimated  channel  state  information,  channel  codes  are  adaptively  assigned,  check  sum  bits  are  added,  and  the  bit  stream 
of  enhancement  layer  is  controlled  in  order  to  meet  the  available  bandwidth.  At  the  receiver  side,  the  transmitted 
corrupted  signals  might  be  corrected  by  channel  decoding  and  residual  errors  are  detected  by  check  sum  bits.  Then, 
channel  state  will  be  estimated  using  finite-state  Markov  chain.  For  video  reconstruction,  corrupted  packets  are  dropped 
and  finally  transmitted  video  sequences  are  reconstructed  using  error  concealment. 


Figure  1:  The  proposed  MIMO  system 


2.1  Scalable  Extension  of  H.264/AVC 


With  the  proposed  video  transmission  scheme  over  the  wireless  heterogeneous  networks,  multi-layered  video  bit 
streams  are  essential  and  thus  created  by  SVC  [3]  in  this  research.  In  this  sub-section,  we  briefly  review  this  video  codec 
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which  is  an  extension  of  the  H.264/AVC  [5] [6]  video  coding  standard.  Traditionally,  there  are  two  different  ways  for 
scalable  video  codec:  either  by  using  a  technique  that  is  intrinsically  scalable  (such  as  bit-plane  arithmetic  coding)  or  by 
using  a  layered  approach.  SVC  supports  a  combination  of  the  two  approaches  so  that  a  full  spatio-temporal  and  quality 
scalable  codec  is  achieved.  A  coded  SVC  video  sequence  consists  of  a  series  of  Network  Abstraction  Layer  (NAL)  units, 
each  containing  the  layer  information.  A  4-byte  SVC  NAL  unit  header,  an  extension  of  H.264/AVC  NAL,  indicates  the 
decoding  dependency  relationship  of  spatial,  temporal,  and  quality  scalability. 


Figure  2:  an  example  of  temporal  decomposition 


Figure  3:  an  example  of  combined  scalability 

The  temporal  scalability  of  SVC  is  typically  given  based  on  the  principle  of  hierarchical  B  pictures.  A  hierarchical 
prediction  structure  with  4  dyadic  hierarchy  stages  (or  4  temporal  scalability  levels)  is  described  in  Figure  2.  In  a  video 
sequence,  the  first  picture  (key  picture)  is  intra-coded  as  IDR  picture  and  next  key  picture  will  be  either  intra-coded  or 
inter-coded  using  previous  key  picture.  All  pictures  between  two  key  pictures  are  hierarchically  predicted  and  encoded. 
Therefore,  a  group  of  picture  (GOP)  is  created  by  a  key  picture  and  all  pictures  that  are  temporally  located  between  a  key 
picture  and  the  previous  key  picture.  Spatial  scalability  is  also  supported  based  on  existing  multi-layered  coding 
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approach.  As  depicted  in  Figure  3,  the  difference  of  spatial  resolution  between  base  layer  and  enhancement  makes  it 
achieve  spatial  scalability. 

For  quality  (or  SNR)  scalability,  SVC  supports  two  types:  coarse  grain  scalability  (CGS)  using  various  inter-layer 
prediction  techniques  and  fine  grain  scalability  (FGS)  known  as  progressive  refinement.  In  this  work,  we  adopt  FGS  for 
SNR  scalability  in  order  to  both  satisfy  each  layer’s  target  bit-rate  and  increase  the  error  robustness.  Within  each  spatial 
resolution  FGS  is  achieved  by  encoding  successive  refinements  of  the  transform  coefficients.  Therefore,  a  picture  is 
represented  by  base  representation  and  FGS  refinement  representations  by  repeatedly  decreasing  the  quantization  step 
size.  The  NAL  units  of  FGS  refinement  layers  can  be  truncated  at  any  arbitrary  point  at  the  encoder  and  thus  error 
robustness  is  increased  by  the  decoder  capable  of  arbitrarily  discarding  corrupted  NAL  unit  streams.  An  example  of 
combined  scalability  with  group  of  picture  (GOP)  8  is  applied  in  experiments  and  described  in  Figure  3.  For  a  more 
detailed  explanation  on  SVC,  see  [3][4] . 


2.2  Finite  State  Markov  Chain 


A  finite-state  Markov  model  is  defined  as  a  finite  set  of  states S  =  {s(),S1,---,SK_1}  ,  where K  is  the  number  of 
states  (Figure  4  shows  an  example  of  2-state  Markov  model).  A  binary  symmetric  channel  (BSC)  with  a  given  crossover 
probability  is  associated  with  each  state Sk,k  G  {0,1,2, •  •  •} ,  so  the  channel  quality  for  each  state  can  be  identified.  Let 


{Sn\S„  gS,h  =  0,1,2  ,  •  •  •}  be  a  constant  Markov  process.  Since  the  constant  Markov  process  has  the  property  of 
stationary  transitions,  the  transition  probability  is  independent  of  time  index  It ,  so 

tj,k  =Pr{S„+i  =sk  K  =Sj}  kj  e  0,1,2,  K-l  (1) 

is  a  constant  KxK  matrix,  which  is  called  the  state  transition  probability  matrix  T  .  The  state  transition  probability 
matrix  is  an  important  parameter  for  a  finite-state  Markov  model.  Note  that  the  sum  of  the  elements  on  each  row  of  T  is 
equal  to  1 . 


K-l 


2>w=l,V*e  {0,1,2,-} 


(2) 


1=0 


Another  set  of  parameters  for  a  finite-state  Markov  model  are  the  crossover  probabilities  in  each  state, 
Ck,k  G  {0,1,2, •••}  .The  last  parameter  for  a  finite-state  Markov  model  is  the  steady  state  probability  of  each  state, 


pk,k  e  {0,1,2,  •  •  •} .  But  the  steady  state  probabilities  are  not  independent.  The  steady  state  probability  pk  for  state 


Sk  can  be  viewed  as  the  probability  of  all  states  transitioning  to  state  Sk  ,  so 


A'-l 


j= « 


=  pk,Vke{0,l,2,--,K-l} 


K-l 

The  functions  only  have  zero  solution.  But  ^  p k  —  1 

k= 0 


(3) 

(4) 


With  any  K  —  1  functions  from  (Eq.  3)  and  the  function  (Eq.  4),  the  pk,k  G  {0,1,2,- •  •} can  be  determined  with  any 
given  state  transition  probability  matrix  T  . 


So  the  K-state  Markov  model  is  decided  by  its  parameters:  state  transition  probability  matrix 
T  =  [tjk],j,ke  [0,1,. ..AT  -  1]  and  crossover  probabilities  ek,k  G  [0,1,..., K  —  1]  .  These  states  and  its  crossover 

probabilities  determine  the  channel  quality  for  any  wireless  links  modeled  by  such  a  Markov  model.  When  an  estimation 
of  the  channel  model  can  be  obtained,  an  adaptive  video  transmission  scheme  can  be  designed  to  take  full  advantage  of 
the  known  channel  parameter  and  the  dynamics  of  the  channel  behavior.  The  parameters  of  the  finite-state  Markov 
channel  model  can  be  obtained  by  collecting  traces  under  various  network  conditions. 
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2.3  Channel  State  Estimation 

We  propose  the  following  channel  state  estimation  at  the  decoder.  The  decoder  estimates  the  channel  state  for  the  next 
one  or  several  transmissions  (according  to  application  latency)  based  on  the  current  received  packet  and  the  packet  after 
channel  decoding.  Each  video  packet  is  protected  by  different  strong  level  of  FEC,  depending  on  the  important  of  video 
coding  layer  and  the  current  channel  state.  After  receiving  one  packet,  the  decoder  first  does  channel  decoding. 

Based  on  the  received  original  packet  and  the  packet  after  channel  decoding,  the  channel  estimation  method  is  as 
following: 

If  the  channel  decoding  is  successful,  we  can  compare  the  received  original  packet  and  the  packet  we  get  after 
decoding.  The  different  number  of  bits  should  be  the  number  of  transmission  error.  The  number  of  transmission  error 
divided  by  the  length  of  the  packet,  is  the  error  probability  pe . 

If  current  state  is  good,  then  the  next  state  is  estimated  as  good; 

Else  if  current  state  is  bad: 

If  pe  <  e0  (the  crossover  probabilities  of  good  state),  then  the  next  state  is  estimated  as  good; 

Else,  the  next  state  is  estimated  as  bad. 

If  channel  decoding  is  unsuccessful,  then  the  next  state  is  estimated  as  bad. 


^01 


Figure  4:  A  two-state  Markov  chain  model 


2.4  Video  Reconstruction 

If  the  bit  errors  are  above  the  capacity  of  the  assigned  channel  coding,  the  transmitted  packets  might  be  corrupted  and 
are  dropped  before  video  reconstruction.  Then,  error  concealment  is  processed  at  the  decoder.  The  detection  of  one  or 
more  missing  pictures  is  done  by  calculating  the  frame  number  gap,  picture  order  count  (POC)  gap,  and  the  GOP  size.  If 
a  base  layer  packet  is  lost,  the  corresponding  enhancement  layer  packet  is  invalid  and  will  be  regarded  as  a  lost  packet 
also.  We  adopt  picture  copy  (PC)  algorithm  in  this  work.  With  this  error  concealment,  each  sample  of  value  of  the 
concealed  picture  is  copied  from  the  corresponding  sample  of  the  first  picture  in  the  reference  list  0.  The  base  layer  PC 
will  only  be  done  when  decoding  a  low  resolution  sequence.  A  lost  enhancement  layer  picture  will  be  concealed  using 
the  first  picture  in  the  reference  picture  list  0.  The  list  0  is  generated  from  the  decoding  process  of  the  enhancement  layer 
and  it  contains  only  high  resolution  pictures. 


3.  EXPERIMENT  RESULTS 

We  provide  numerical  examples  to  show  how  the  proposed  video  transmission  scheme  with  adaptive  channel  coding 
is  able  to  achieve  stable  reconstructed  video  over  time-varying  wireless  channel. 

3.1  Performance  of  Channel  State  Estimation 
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Using  the  proposed  method  above,  we  conduct  the  prediction  experiments  based  on  different  Markov  models.  We 
tested  different  state  transition  matrix.  In  all  cases,  the  crossover  probabilities  of  the  good  state  is  0.1%;  and  the 
crossover  probabilities  of  the  bad  state  is  10%.  Table  1  gives  the  prediction  error  with  different  Markov  models.  From 
the  table  we  can  see  that  when  the  transition  probabilities  between  the  states  become  larger,  the  prediction  error  also 
becomes  larger.  This  is  because  when  the  transition  probabilities  between  the  states  are  large,  the  channel  becomes 
“unstable”,  it  is  not  easy  to  predict  the  next  state  based  on  previous  states. 

Table  1.  Prediction  errors  with  different  Markov  models 


State  transition  matrix 

Prediction  Error  (%) 

[0.99875  0.00125] 

[  0.005  0.995  J 

0.202 

"0.99  O.Of 
0.02  0.98_ 

1.35 

'  0.98  0.025 

0.025  0.97 

2.48 

3.2  Performance  of  Reconstructed  Video  PSNR 

In  this  sub-section,  we  conduct  experiments  to  show  the  performance  of  the  proposed  system  with  reconstructed 
PSNR  of  the  decoded  video  sequences.  As  shown  in  sub-section  3.1,  the  prediction  of  channel  state  (CS)  estimation 
would  be  quite  accurate  if  the  transition  probabilities  between  the  states  become  large.  Thus,  in  this  sub-section,  we 
conduct  an  experiment  with  ideal  case  that  the  channel  state  estimation  is  perfect  and  no  delay.  Three  video  sequences, 
‘Mobile’,  ‘Carphone’,  and  ‘Foreman’,  are  tested.  All  test  sequences  are  64  frames  with  CIF  15[Hz]  and  encoded  by 
scalable  H.264/AVC  reference  model  JSVM  7.0  [4]  to  generate  2-layer  scalable  video  bit-streams  with  GOP  size  8. 
Reed-Solomon  (RS)  Codes  [7]  are  adopted  to  protect  the  transmitted  bit-streams  since  it  maintains  maximum  erasure 
protection  while  produces  a  minimum  of  redundancy.  We  assume  that  640  [Kbits/sec]  is  available  as  data  rate  and  then 
256[Kbits/sec]  is  assigned  for  base  layer  and  384[Kbits/sec]  is  utilized  for  enhancement  layer.  Then,  in  order  to  achieve 
unequal  error  protection  (UEP),  different  channel  coding  redundancies  are  assigned  for  base  and  enhancement  layers. 
Fixed  UEP  RS  codes  are  assigned  in  no  CS  feedback  system  while  assigning  adaptive  UEP  RS  codes  in  the  proposed 
system  that  has  the  feedback  of  CS.  Note  that  the  enhancement  layer  bit  streams  are  adaptively  dropped  to  adjust  the  RS 
channel  coding  according  to  the  channel  state  so  as  to  achieve  the  adaptive  transmission  scheme  with  the  fixed  available 
data  rate 

The  average  reconstructed  PSNR  (luminance  component)  for  ‘Mobile’,  ‘Foreman’,  and  ‘Carphone’  are  shown  in 
Figure  5,  Figure  6,  and  Figure  7,  respectively.  In  these  figures,  we  clearly  demonstrate  that  the  proposed  adaptive 
transmission  scheme  (with  CS  Feedback)  based  on  channel  state  estimation  achieves  steady  reconstructed  video 
sequences  over  time-varying  wireless  networks  as  compared  with  non-rate  adaptive  source  coding  system  (w/o  CS 
Feedback).  In  table  2,  we  illustrated  the  average  reconstructed  PSNR  of  three  test  sequences. 


Table  2.  Average  reconstructed  PSNR  of  test  sequences 


Mobile 

Carphone 

Foreman 

w/o  CS  Feedback 

26.13  [dBl 

33.76  [dBl 

28.35  [dBl 

with  CS  feedback 

32.53  TdBl 

37.42  [dBj 

35.82  rdBl 

4.  CONCLUSION 

We  propose  in  this  research  an  adaptive  video  transmission  based  on  H.264  scalable  video  coding  and  the  adaptive 
channel  estimation  based  on  finite-state  Markov  model  (See  Figure  4).  The  H.264  scalable  video  coding  enables  the  rate 
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adaptive  source  coding  and  the  feedback  of  channel  parameters  facilitates  the  adaptive  channel  coding  based  on  the 
dynamics  of  the  channel  behavior.  As  a  result,  we  are  able  to  develop  a  true  adaptive  joint  source  and  channel  based  on 
instantaneous  channel  estimation  feedback.  Preliminary  experimental  results  demonstrate  that  the  estimation  of  the 
finite-state  Markov  channel  can  be  quite  accurate  and  the  adaptive  video  transmission  based  on  channel  estimation  is 
able  to  perform  significantly  better  than  the  simple  channel  model  in  which  only  average  bit  error  rate  is  used  for  joint 
source  and  channel  coding  design.  The  proposed  methods  can  be  easily  extended  to  multi-state  Markov  model  based  on 
different  application  scenarios. 
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Figure  5:  Reconstructed  PSNR  of  Mobile 
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Figure  6:  Reconstructed  PSNR  of  Carphone 


Figure  7:  Reconstructed  PSNR  of  Foreman 
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